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DETAILED ACTION 
Drawings 

1 . Figures 2 and 15 should be designated by a legend such as -Prior Art- because 
only that which is old is illustrated. See MPEP § 608.02(g). Corrected drawings in 
compliance with 37 CFR 1 .121(d) are required in reply to the Office action to avoid 
abandonment of the application. The replacement sheet(s) should be labeled 
"Replacement Sheet" in the page header (as per 37 CFR 1 .84(c)) so as not to obstruct 
any portion of the drawing figures. If the changes are not accepted by the examiner, the 
applicant will be notified and informed of any required corrective action in the next Office 
action. The objection to the drawings will not be held in abeyance. 

Claim Objections 

2. Claim 3 is objected to because of the following informalities: the phrase 
"according to claims 1" should read "according to claim 1". Appropriate correction is 
required. 

3. Claim 5 is objected to because of the following informalities: the phrase 
"according to any of claims 1" should read "according to claim 1". Appropriate 
correction is required. 
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4. Claim 6 is objected to because of the following informalities: the preamble of 
claim 6 indicates that the claim is an independent claim, however the body of claim 6 
indicates that the claim is a dependant claim by referencing preceding claims. 
Examiner considered claim 6 as if the claim was independent. Appropriate correction is 
required. 

Claim Rejections - 35 USC § 101 

5. 35 U.S.C. 101 reads as follows: 

Whoever invents or discovers any new and useful process, macliine, manufacture, or composition of 
matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the 
conditions and requirements of this title. 

6. Claims 1-5 are rejected under 35 U.S.C. 101 as not falling within one of the four 
statutory categories of invention. Supreme Court precedent and recent Federal Circuit 
decisions indicate that a statutory "process" under 35 U.S.C. 101 must (1) be tied to 
another statutory category (such as a particular apparatus); or (2) transform underlying 
subject matter (such as an article or material) to a different state or thing. While instant 
claims 1-5 recite a series of steps or acts to be performed, the claim neither transforms 
underlying subject matter nor is positively tied to another statutory category that 
accomplishes the claimed method steps, and therefore does not qualify as a statutory 
process. For example, it is unclear what performs, in electronic form the determining, 
selecting, accumulating, testing, outputting, comparing and interpolating steps recited in 
the method claims. 



Application/Control Number: 10/535,420 Page 4 

Art Unit: 2621 

7. Claims 6-1 1 are rejected under 35 U.S.C. 101 as not falling within one of the four 
statutory categories of invention. Supreme Court precedent and recent Federal Circuit 
decisions indicate that a statutory "process" under 35 U.S.C. 101 must (1) be tied to 
another statutory category (such as a particular apparatus); or (2) transform underlying 
subject matter (such as an article or material) to a different state or thing. While instant 
claims 6-1 1 recite a series of steps or acts to be performed, the claim neither transforms 
underlying subject matter nor is positively tied to another statutory category that 
accomplishes the claimed method steps, and therefore does not qualify as a statutory 
process. For example, It is unclear what performs, in electronic form the determining, 
generating, selecting, accumulating, warping and calculating steps recited in the method 
claims. 

8. Claim 1-11 are rejected under 35 U.S.C. 101 because invention is directed 
towards non-statutory subject matter (Please see the MPEP 2106 Section IV. 
Determine Whether the Claimed Invention Complies with 35 U.S.C. 101). The 
specification at page 1 1 , line 33 - page 12, line 4, indicates that the invention can be 
implemented in software. Since the software is not embodied within a computer 
readable medium, the invention is directed towards non-statutory subject matter. 

Claim Rejections - 35 USC § 102 

9. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 1 02 that 
form the basis for the rejections under this section made in this Office action: 
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(e)the invention was described in (1) an application for patent, publislied under section 122(b), by 
another filed in tlie United States before the invention by the applicant for patent or (2) a patent 
granted on an application for patent by another filed in the United States before the invention by the 
applicant for patent, except that an international application filed under the treaty defined in section 
351(a) shall have the effects for purposes of this subsection of an application filed in the United States 
only if the international application designated the United States and was published under Article 21(2) 
of such treaty in the English language. 

10. Claims 1-4, and 12-17 are rejected under 35 U.S.C. 102(e) as being anticipated 
by Subramaniyan et al. (US Patent Application Publication # 2004/0028134 A1). 

1 1 . Regarding claim 1 , Subramaniyan discloses: 

A method of global motion estimation between frames of a motion-compensated inter- 
frame encoded video sequence, each frame of the sequence having a plurality of 
motion vectors encoded therein relating the frame to a preceding and/or succeeding 
frame of the sequence, the method comprising, for a particular frame: 

a) determining a motion estimation representative of the global motion between 
the particular frame and its anchor frame on the basis of motion vectors therebetween 
([0035]; the motion estimation circuit 1 10 can determine the global motion vector by 
using an average of all final motion vectors in a previous frame ): 

b) determining one or more further motion estimations representative of the 
global motion between the particular frame and its anchor frame at least partially on the 
basis of motion vectors between the particular frame and one or more preceding or 
succeeding other frames ([0035]; [0026], motion estimation is computed for blocks of 
image data from a current image frame using one or more previously processed image 
frames ): and 
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c) selecting one of the motion estimations wliicli meets at least one 
predetermined criterion as being representative of the global motion of the frame 
([0035], determining the global motion vector based on the each of the final motion 
vectors in a previous frame with a difference metric that is below the threshold; [0049], 
This process continues until one of 3 conditions are met: [0050], CONDITION 1 : The 
MSAD is below a threshold THRESH4 . given by: THRESH4=A*Q+B). 

12. Regarding claim 2, Subramaniyan discloses: 

A method according to claim 1 , wherein the determining step b) further comprises the 
steps of: 

e) determining one or more motion estimations representative of the global 
motion of the frame with respect to one or more respective preceding or succeeding 
other frames ([0035]; [0026], motion estimation is computed for blocks of image data 
from a current image frame using one or more previously processed image frames ): 

f) determining one or motion estimations respectively representative of the global 
motion of the one or more other frames with respect to the anchor frame ([0035]; the 
motion estimation circuit 110 can determine the global motion vector by using an 
average of all final motion vectors in a previous frame ): and 

g) accumulating the respective motion estimations to give one or more respective 
overall motion estimations each substantially representative of the global motion of the 
frame with respect to the anchor frame ([0035], the motion estimation circuit 110 can 
determine the global motion vector by using an average of all final motion vectors in a 
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13. Regarding claim 3, Subramaniyan discloses: 

A method according to claims 1 , wherein the selecting step c) further comprises 

testing the motion estimations in turn ([0035], comparing the difference metric for 
each of the final motion vectors in the previous frame with a predetermined threshold; 
[0049], This process continues until one of 3 conditions are met: [0050], CONDITION 1 : 
The MSAD is below a threshold THRESH4 . given by: THRESH4=A*Q+B); and 

outputting a motion estimation as being representative of the global motion of the 
frame If It passes the test, wherein the test is applied in turn to motion estimations once 
they have been determined ([0035], determining the global motion vector based on the 
each of the final motion vectors in a previous frame with a difference metric that is 
below the threshold; [0049], This process continues until one of 3 conditions are met: 
[0050], CONDITION 1 : The MSAD is below a threshold THRESH4 . given by: 
THRESH4=A*Q+B), and if the test is passed then no further motion estimations are 
determined ([0049], This process continues until one of 3 conditions are met : [0050], 
CONDITION 1 : The MSAD is below a threshold THRESH4, given by: 
THRESH4=A*Q+B). 

14. Regarding claim 4, Subramaniyan discloses: 

A method according to claim 3, wherein the test comprises 
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comparing tlie motion estimation with a threshold value ([0049]-[0050], 
THRESH4), wherein the test is passed if the parameters of the motion estimation do not 
exceed the threshold value ([0049]-[0050], [0054] The best MV at the end of the second 
stage is chosen as the best MV for the macroblock; the MV having a MSAD below 
THRESH4 is chosen as the best MV) 

15. Regarding claim 12, Subramaniyan discloses: 

A computer program or suite of programs arranged such that when executed on 
a computer system the program or suite of programs causes the computer system to 
perform the method of claim 1 ([0085]). 

16. Regarding claim 13, Subramaniyan discloses: 

A computer readable storage medium storing a computer program or suite of 
programs according to claim 12 ([0085]). 

1 7. Regarding claim 1 4, Subramaniyan discloses the system limitations of this claim 
as discussed above with respect to claim 1 . 

18. Regarding claim 1 5, Subramaniyan discloses the system limitations of this claim 
as discussed above with respect to claim 2. 
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19. Regarding claim 16, Subramaniyan discloses the system limitations of this claim 
as discussed above with respect to claim 3. 

20. Regarding claim 1 7, Subramaniyan discloses the system limitations of this claim 
as discussed above with respect to claim 4. 

21 . Claims 6-9 are rejected under 35 U.S.C. 102(e) as being anticipated by Jinzenji 
et al. (US Patent # 6,977,664 B1). 

22. Regarding claim 6, Jinzenji discloses: 

A method of generating panoramic images from a motion-compensated inter-frame 
encoded video sequence, the method comprising: 

for each frame of the sequence, determining the global motion of each frame with 
respect to its anchor frame using the method of any of the preceding claims (Jinzenji col 
8, line 32-56; global motion is obtained in step 1); and 

generating at least one panoramic image representing the frames of the video 
sequence using the global motion estimations thus determined (Jinzenji col 8, line 32- 
56; a provisional sprite <panoramic image> is generated). 

23. Regarding claim 7, Jinzenji discloses: 

A method according to claim 6, wherein the generating step further comprises: 
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selecting a particular frame of the sequence as a reference frame, the plane of 
the reference frame being a reference plane (Jinzenji col 8, line 32-56; the reference 
coordinate system which is for the reference frame); 

for each frame other than the reference frame, accumulating the global motion 
estimations from each frame back to the reference frame (Jinzenji col 8, line 32-56; 
each original image of the arbitrary frames is mapped to a reference coordinate system 
which is for the reference frame); 

warping each frame other than the reference frame onto the reference plane 
using the accumulated global motion estimations to give one or more pixel values for 
each pixel position in the reference plane (Jinzenji col 8, line 32-56; frame warping 
occurs when frames are mapped to the reference coordinate system using global 
motion so as to insert or overwrite pixels); and 

for each pixel position in the reference plane, selecting one of the available pixel 
values for use as the pixel value in the panoramic image (Jinzenji col 8, line 32-56; a 
pixel value of a point is obtained from pixel values which exist in the same point). 

24. Regarding claim 8, Jinzenji discloses: 

A method according to claim 7, wherein the selecting step comprises selecting a 
substantially median pixel value from the available pixel values for use in a background 
panoramic image (Jinzenji col 10, line 8-1 1 ; for a plurality of pixels which are mapped to 
the same coordinates, a median value of the pixels is selected as the value of the 
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25. Regarding claim 9, Jinzenji discloses: 

A method according to claim 7, wherein the selecting step comprises selecting a 

substantially most different pixel value from the available pixel values for use in a 
foreground panoramic image (Jinzenji col 8, line 47-51; using a threshold to select the 
most different pixel). 

Claim Rejections - 35 USC § 103 

26. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

27. Claims 5 and 18 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Subramaniyan et al. (US Patent Application Publication # 2004/0028134 A1 ) in view of 
Lee et al. (US Patent Application Publication # 2003/0103568 Al). 

28. Regarding claim 5, 
Subramaniyan teaches: 

A method according to any of claims 3 (as shown above), 
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Subramaniyan does not teach: 

wherein if the test is failed, the method further comprises: 

interpolating between the motion estimations of adjacent frames to give an 
interpolated motion estimation which is then output as the motion estimation 
representative of the global motion of the frame. 

Lee teaches: 

wherein if the test is failed, the method further comprises: 

interpolating between the motion estimations of adjacent frames to give an 
interpolated motion estimation which is then output as the motion estimation 
representative of the global motion of the frame (Lee: [0061]-[0063]). 

At the time of invention, it would have been obvious to a person having ordinary 
skill in the art to combine the teachings of Lee with Subramaniyan. The teachings of 
Lee provide for motion compensated interpolation that eliminates blocking artifacts (Lee 
[0048]), thereby increasing the ability of Subramaniyan to generate accurate global 
motion in the presence of blocking artifacts. 

29. Regarding claim 18, Subramaniyan in view of Lee discloses the system 
limitations of this claim as discussed above with respect to claim 5. 
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30. Claims 10-1 1 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Jinzenji et al. (US Patent # 6,977,664 B1 ) in view of Szeliski et al. (US Patent # 
6,348,918 61). 

31 . Regarding claims 10-11, 
Jinzenji teaches: 

A method according to claim 7 (as shown above), 

Jinzenji fails to teach: 

wherein the selecting step comprises: 

calculating the mean pixel value of the available pixel values; 

calculating the LI distance between each available pixel value and the calculated 
mean pixel value; and 

select the pixel value with the median LI distance for use in a background 
panoramic image. 

select the pixel value with the maximum LI distance for use in a foreground 
panoramic image. 

Szeliski teaches: 

wherein the selecting step comprises: 

calculating the mean pixel value of the available pixel values (Szeliski col 8, line 
57-65; taking the mean of the color or intensity values); 
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calculating the LI distance between each available pixel value and the calculated 
mean pixel value (Szeliski col 8, line 57-65; where the averaging is weighted by the 
distance of each pixel from the nearest invisible pixel); and 

select the pixel value with the median L1 distance for use in a background 
panoramic image (Szeliski col 8, line 57-65; using the median technique). 

select the pixel value with the maximum LI distance for use in a foreground 
panoramic image (Szeliski col 8, line 57-65; the simplest technique is the median 
technique, but many others exist. This portion of Szeliski discloses blending specifically 
for a background image. This portion of Szeliski also discloses blending generally. 
Instead of using the median technique for blending background pixels, a maximum 
technique is obvious for blending foreground pixels. This is because foreground pixels 
are most different from background pixels). 

It would have been obvious to a person having ordinary skill in the art to combine 
the teachings of Szeliski with Jinzenji. Using the blending technique of Szeliski would 
smooth out disparities of the panoramic image of Jinzenji, thus creating a panoramic 
image with increased image quality (Szeliski col 9, line 6-8). 

32. Claims 19-22 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Subramaniyan et al. (US Patent Application Publication # 2004/0028134 A1) in view of 
Jinzenji et al. (US Patent # 6,977,664 B1). 
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33. Regarding claim 19, 

Subramaniyan teaches: 

A system for generating panoramic images from a motion-compensated inter- frame 
encoded video sequence, comprising: 

a system for global motion estimation between frames of a motion-compensated 
inter-frame encoded video sequence as claimed in claim 14, and further arranged to 
provide global motion estimations for each frame (Subramaniyan: [0035]; the motion 
estimation circuit 1 10 can determine the global motion vector by using an average of all 
final motion vectors in a previous frame ): 

Subramaniyan fails to teach: 

panoramic image generating means for generating at least one panoramic image 
representing the frames of the video sequence using the global motion estimations thus 
determined. 

Jinzenji teaches: 

panoramic image generating means for generating at least one panoramic image 
representing the frames of the video sequence using the global motion estimations thus 
determined (Jinzenji col 8, line 32-56; a provisional sprite <panoramic jmage> is 
generated). 



Application/Control Number: 10/535,420 Page 16 

Art Unit: 2621 

It would have been obvious to a person having ordinary skill in the art to combine 
the teachings of Jinzenji with Subramaniyan. Using the panoramic generating means of 
Jinzenji would allow for creation of a panoramic image based on the global motion of 
Subramaniyan, thereby allowing a user to implement an established use for global 
motion. 

34. Regarding claim 20, 
Jinzenji teaches: 

wherein the panoramic image generating means is further arranged in use to: 

select a particular frame of the sequence as a reference frame, the plane of the 

reference frame thereby being a reference plane (Jinzenji col 8, line 32-56; the 

reference coordinate system which is for the reference frame); 

for each frame other than the reference frame, accumulate the global motion 

estimations from each frame back to the reference frame (Jinzenji col 8, line 32-56; 

each original image of the arbitrary frames is mapped to a reference coordinate system 

which is for the reference frame); 

warp each frame other than the reference frame onto the reference plane using 

the accumulated global motion estimations to give one or more pixel values for each 

pixel in the reference plane (Jinzenji col 8, line 32-56; frame warping occurs when 

frames are mapped to the reference coordinate system using global motion so as to 

insert or overwrite pixels); and 
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for each pixel position in the reference plane, select one of the available pixel 
values for use as the pixel value in the panoramic image (Jinzenji col 8, line 32-56; a 
pixel value of a point is obtained from pixel values which exist in the same point). 

35. Regarding claim 21 , 
Jinzenji teaches: 

wherein the panoramic image generating means is further arranged to select a 
substantially median pixel value from the available pixel values for use in a background 
panoramic image (Jinzenji col 10, line 8-1 1 ; for a plurality of pixels which are mapped to 
the same coordinates, a median value of the pixels is selected as the value of the 
coordinates of the provisional sprite). 

36. Regarding claim 22, 
Jinzenji teaches: 

wherein the panoramic image generating means is further arranged to select a 
substantially most different pixel value from the available pixel values for use in a 
foreground panoramic image (Jinzenji col 8, line 47-51; using a threshold to select the 
most different pixel). 

37. Claims 23-24 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Subramaniyan et al. (US Patent Application Publication # 2004/0028134 A1) in view of 
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Jinzenji et al. (US Patent # 6,977,664 B1) as applied to claim 19 above 
view of Szeliski et al. (US Patent # 6,348,91 8 B1 ). 

38. Regarding claim 23 and 24, 
Subramaniyan in view of Jinzenji teaches: 
A system according to claim 19 (as shown above), 

Subramaniyan in view of Jinzenji fails to teach: 
wherein the panoramic image generating means is further arranged to: 

calculate the mean pixel value of the available pixel values; 

calculate the LI distance between each available pixel value and the calculated 
mean pixel value; and 

select the pixel value with the median LI distance for use in a background 
panoramic image. 

select the pixel value with the maximum LI distance for use in a foreground 
panoramic image. 

Szeliski teaches: 

wherein the panoramic image generating means is further arranged to: 

calculate the mean pixel value of the available pixel values (Szeliski col 8, line 
57-65; taking the mean of the color or intensity values); 
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calculate the LI distance between each available pixel value and the calculated 
mean pixel value (Szeliski col 8, line 57-65; where the averaging is weighted by the 
distance of each pixel from the nearest invisible pixel); and 

select the pixel value with the median L1 distance for use in a background 
panoramic image (Szeliski col 8, line 57-65; using the median technique). 

select the pixel value with the maximum LI distance for use in a foreground 
panoramic image (Szeliski col 8, line 57-65; the simplest technique is the median 
technique, but many others exist. This portion of Szeliski discloses blending specifically 
for a background image. This portion of Szeliski also discloses blending generally. 
Instead of using the median technique for blending background pixels, a maximum 
technique is obvious for blending foreground pixels. This is because foreground pixels 
are most different from background pixels). 

It would have been obvious to a person having ordinary skill in the art to combine 
the teachings of Szeliski with Subramaniyan in view of Jinzenji. Using the blending 
technique of Szeliski would smooth out disparities of the panoramic image of 
Subramaniyan in view of Jinzenji, thus creating a panoramic image with increased 
image quality (Szeliski col 9, line 6-8). 

Double Patenting 

39. The nonstatutory double patenting rejection is based on a judicially created 
doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the 
unjustified or improper timewise extension of the "right to exclude" granted by a patent 
and to prevent possible harassment by multiple assignees. A nonstatutory 
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obviousness-type double patenting rejection is appropriate where the conflicting claims 
are not identical, but at least one examined application claim is not patentably distinct 
from the reference claim(s) because the examined application claim is either anticipated 
by, or would have been obvious over, the reference claim(s). See, e.g.. In re Berg, 140 
F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 
USPQ2d 2010 (Fed. Cir. 1993); In re Long!, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 
1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 
F.2d 438, 164 USPQ 619 (CCPA 1970); and In re Thorington, 418 F.2d 528, 163 
USPQ 644 (CCPA 1969). 

A timely filed terminal disclaimer in compliance with 37 CFR 1 .321 (c) or 1 .321 (d) 
may be used to overcome an actual or provisional rejection based on a nonstatutory 
double patenting ground provided the conflicting application or patent either is shown to 
be commonly owned with this application, or claims an invention made as a result of 
activities undertaken within the scope of a joint research agreement. 

Effective January 1 , 1994, a registered attorney or agent of record may sign a 
terminal disclaimer. A terminal disclaimer signed by the assignee must fully comply with 
37 CFR 3.73(b). 



40. Claim 6-1 1 are provisionally rejected on the ground of nonstatutory obviousness- 
type double patenting as being unpatentable over claims 11-16 of copending 
Application No. 10/535,621. Although the conflicting claims are not identical, they are 
not patentably distinct from each other because Application No. 10/535,621 discloses 
claims 6-1 1 ('621 claims 11-16; '621 pg 24, line 1 6 - pg 27 line 21 ). 

This is a provisional obviousness-type double patenting rejection because the 
conflicting claims have not in fact been patented. 



Conclusion 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to JAMES PONTIUS whose telephone number is (571 ) 
270-7687. The examiner can normally be reached on Monday - Thursday, 8 AM - 4 PM 
est.. 
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If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Mehrdad Dastouri can be reached on (571) 272-7418. The fax phone 
number for the organization where this application or proceeding is assigned is 571- 
273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retheval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 



/J. P./ 

Examiner, Art Unit 2621 
/Andy S. Rao/ 

Acting Supervisory Patent Examiner, Art Unit 2621 
July 6, 2009 



